Lossless coding through the concentration of measure phenomenon

The lossless codification of stationary memoryless binary sources using block coding is studied with the additional constraint of requiring only two codeword lengths. The fundamental coding scheme of Shannon based on the asymptotic equipartition property shows the existence of block codes of this type with average codeword lengths per source output converging to the binary entropy as the block length n goes to infinity. We show that the average codeword length per source output for such block codes converges to the binary entropy at a rate slower than O ( n−1/2 ) . Nevertheless, for every > 0, we construct an appealing coding procedure with two possible codeword ∗Both authors would like to thank CIMAT, Gto, Mexico, for its warm hospitality while part of this research was done. †Research supported in part by a NSF grant. CH would also like to thank the Laboratoire de Probabilités et Modèles Aléatoires, Université de Paris VI for its warm hospitality while part of this research was conducted.